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An  autoregressive  process  {y  } of  order  p with  mean  0 is 

X 

defined  by 


(1) 


yt  + ^lyt-l  + + ^pyt-p  Ut  s >0,1,.. 


where  the  u^_  are  independent  random  variables  with  ^u^  = 0,Qu^  = ^ » 
2 

0 < a < °°  . The  stochastic  process  is  stationary  and  y is  indepen- 

X 


den*  of  ut+1,  ut+2, 


if  and  only  if  the  3 are  such  that  the 

J 


associated  polynomial  equation 


(2) 


b (w)  = l 3.  wP_j  = 0 


j=0 


where  3n  = 1 > has  roots  w_ , . ..,  w less  than  1 in  absolute  value. 
O’  1 p 

The  purpose  of  this  paper  is  to  show  that  the  generalized  variance  of  the 

process  is  a power  of  the  variance  of  u times  H?  (l-w.w.)  1 . 

X 1 s J -L  1 J 

The  covariance  sequence  of  the  process  is  composed  of 


a = Gy.  y.  -a  3 s - 0,1,..  . . Consider  a sequence  of  observations 
S X X*rS  — S 

yx>  . ..,  yT  for  Tip  constituting  a sample  vector  yT  = • 


The  covariance  matrix  of  the  sample  vector  is 


(3) 


“ ^°i-j  ^ _ ~T  ^ ° §1 


The 


i i 2 T 

determinant  = (a  ) |Q^|  is  the  generalized  variance  of  y 

In  the  Gaussian  case  the  joint  density  of  y^  and 

Up+1’  • * • . UT  (T  ^ P)  is 


T 


(4) 


Q. 


-l 


1 

2 


:2tt)T/2  aT 


exp< 


2a2  L' 


y'  Q 1 y + l 
■p  ~p  ~p  t=p+1  t 


T 


Since  the  Jacobian  of  the  transformation  from  y , u um  to 

ip  p+1  ’ T 

y^  is  1 , the  constant  of  the  density  of  y^  is  the  same  as  of  (4) 


and  hence 


Since  1^1 


IQ-1!  = | Q"1 I , T > p 


(See  Walker  (l96l)  and  Siddiqui  (1958),) 


for  T >.  p , we  call  |Q  | the  normalized  genera- 

~ Jr 


lized  variance  of  the  process . 


For  T J>  p substitution  for  u from  (l)  , t = p+l,...,T,  into 

u 

t 

(4)  to  obtain  the  density  of  y^  yields  the  quadratic  form  y^  y , 
showing  that  every  element  of  is  a second  degree  polynomial  in 


^1  9 * * * 

density  of  y 


. except  possibly  elements  of 


-1 


However,  since  the 


T 


is  identical  to  the  density  of  y^,  . 


'1  » 


the  elements  of  Q ^ must  be  second  degree  polynomials  in  fL  0 

~P  P 

The  components  of  are  therefore  polynomials  in  the  roots 

of  degree  at  most  2p  . Hence,  the  determinant  |Q  2|  is  a polynomial 

~P 

in  wp,  . w of  degree  at  most  (2p)^  . 
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Lemma  1.  If 


(5)  C= ( c . ) 

ij 


then 


(6) 

|c|  = 

n (h 

- 

i<j  J 

and 

(7) 

°1*  - 

( 11  h) 
Wk 

n (h  - h ) , 

i<  j J 

where  C„,  denotes  the  cofactor  of  c, , in  C . 

— Ik ik  — ~ 

Proof.  C is  a Vandermonde  matrix,  and  |c|  and  C ^ are  given, 

for  example,  hy  Hamming  (1962),  Sections  8.2  and  10.3.  A direct  proof 

of  (7)  using  (6;  is  as  follows:  to  form  C , delete  row  1 and  column 

k of  | C | ; in  the  cofactor,  factor  h.  out  of  the  i-th  column  (i^k) 

~ 1 

to  obtain  a Vandermonde  determinant  of  order  p-1  . Q.E.D. 

Lemma  2.  The  determinant  of  order  p , 


(8) 


D = 
P 


1 


a . + b . 
1 J 


II  (a  - a.  ) (b  - b ) 
i<j  J J 


P 

IT  (a.  + b . ) 

i.3-1  1 3 
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Proof.  This  is  Cauchy's  determinant;  see,  for  example.  Bellman 


(i960).  Section  11.6,  Exercise  1.  A direct  proof  is  as  follows:  To 

convert  into  0 each  element  in  the  first  column,  except  for  that 
in  the  first  row,  we  subtract  from  each  row  an  appropriate  multiple 
of  its  first  row.  The  i,j-th  element  is  thus  converted  into 


(9) 


1 


a.+b. 

1 5 


1 


a,  +b  . 
1 J 


ai+bi 

a.+b, 
1 1 


9/ . — Si-, 

1 1 

a.+b, 
1 1 


b . -b 
1 1 

a,  +b . a.+b. 
1 J 1 J 


The  first  factor  on  the  right-hand  side  is  common  to  tjie  i-th  row  and 
the  second  to  the  j-th  column.  Hence, 


(10) 


D 

P 


Vbi 


a.  -a, 
1 1 

a.+b, 
1 1 


vh  / 
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and  the  result  follows.  Q.E.D. 

Theorem.  For  , . . . , g such  that  the  roots  of  (2)  are  less 
than  1 in  absolute  value  for  T >.  p 


(11) 


|£t|  = (a2)T_  (1-w^)-1 

i s j — 1 


Proof.  ¥e  first  consider  the  case  where  w, , ...,  w are  dif- 
1'  p 


ferent  and  different  from  0 . If  h.  = w. 

1 x 


-1 


then  in  (5)  |c|  ^ 0 

Anderson  (1971)  in  (24)  of  Section  5.3  gives  an  expression  for  the 


elements  of  Z in  terms  of  w,  , . . . , w 
~P  1 P 


(See  also  Problem  19  of 


Chapter  5.)  Then 
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(12) 


k 


2 2 

Cn  •••  Ci 

-I! iE. 


|c|2p 


-2 


|v|  , 


where 


(13)  |V 


-1 

1 

w. 

1 

1 

1 

1 - w.w. 
1 J 

—1 

w . + ( -w . ) 

P 

n w. 
i-1  1 

W.  + (-W.) 

1 J 

1 J 

n (w.1  - w.1) (w.  - w. ) 
_1 j<,i  J 1 1 J 


P 

n w. 


1 n (w.1  - w. ) 

i=l  i» j=l  1 J 


■ <jw 


n (w.  - w )2  wTk.1 

P K.i  1 J 1 J 

, P 

II  (l  - w. w . ) 

i,j=l  1 J 


n (w.  - 


= -111 


w.  y 

J 


p 

n (i 

i,j=l 


w.w. ) 
i J 


Further  5 


(lU) 


2 2 

Cfh  ...  cf 

-I! 


C 


2p-2 


[ 


{ n h? n 
W 1 i i<i 


(h.  - h.) 
J i 


P-2 


T2 


n (h.  - h.) 

i<J  3 1 


P-1 


n h2l>-2 

i=l  1 


n (h  - h. )2  n (w.  - w. )2 
i< j J 1 i<J  1 J 
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and  (ll)  follows  for  the  roots  different  and  nonzero.  The  determinant 

K1!  is  the  polynomial  H?  . , (l  - w.v.)  , which  holds  for  all  w. 
~P  i5J-l  i J j 

such  that  |w.|  < 1,  j = 1,  . p . Q.E.D. 


Discussion 

1.  If  the  process  is  Gaussian,  the  normalizing  constant  in  the 


normal  density  of  y is  (2tt) 

~ i 


-T/2 


times 


(15) 


_1 

2 


<a2rT/2  n (1- 

ij=l 


w.  w. ) 
1 J 


1 

2 


2.  If  one  or  more  of  the  roots  approaches  1 in  absolute  value, 

I 0 an<i  |2t|  00  • These  facts  agree  with  the  nonexistence  of 

a nontrivial  stationary  process  satisfying  (l)  if  one  or  more  roots  are 
equal  to  1 in  absolute  value. 

3.  Grenander  and  Szego  (1958)  in  effect  showed  that 

Igfl  = ^i,j=l  ^ ~ wiwj  ^ 1 "hy  use  of  an  integral  of  |b(w)|~2  , 
though  they  did  not  relate  this  result  to  the  generalized  variance  of 
the  autoregressive  process.  Walker  (1961)  noted  that 

| Qrp | = |Qp|  = 1/ 1 Q”1 1 for  T > p,  An  alternate  proof  of  the  theorem  can 
be  assembled  from  the  results  of  Grenander  and  Szego  and  Walker.  (See 
also  Finch  (i960).) 

4.  A moving  average  model  of  order  q is  defined  by 


(16) 


Xt  Vt  + alVt-l  + • ' • + aqVt-q  » 


6 


0 


where  the 


v^_  are  independent  random  variables  with  = 

2 

0 < t < 00  . The  associated  polynomial  equation 


(17) 


a^z^  1 + 


+ a = 0 
q. 


has  roots  z^,  z^  . Durbin  (1959)  conjectured  that  if  is 

the  covariance  matrix  of  x , . . . , x.^  generated  by  (l6),  and  if  all 
roots  of  (2)  are  less  than  1 in  absolute  value,  then 


(18) 


lim  |N J 


for  some  n sufficiently  large  compared  with 
j = 1,  p . Finch  (i960)  showed  that 


p=q  when 


a =3, 


3 


(19) 


lim  | W | = lim  | Q | 

1400  ~ T4oo  ^ 


by  use  of  some  results  of  Grenander  and  Szego  (1958)  and  gave  explicitly 
the  limiting  value  of  the  generalized  variance  for  an  autoregressive 
moving  average  process.  Walker  (1961)  used  more  algebraic  methods  to 

show  (l8)  for  n = p = q . As  an  example,  these  results  for  p = q = 1 

P Pl+P  P P 

are  Q = 1/(1-$*)  and  |N  | = (l-a^  )/(l-OLp  -*  l/(l-a  ) as  T ->  » . 

Durbin  (1959)  considered  the  case  p = q = 2 in  detail. 

For  further  discussion,  see  the  recent  paper  by  Shaman  (1976). 
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